Scaling Reinforcement Learning Paradigms for Motor Control

Reinforcement learning offers a general framework to explain reward related learning in artificial and biological motor control. However, current reinforcement learning methods rarely scale to high dimensional movement systems and mainly operate in discrete, low dimensional domains like game-playing, artificial toy problems, etc. This drawback makes them unsuitable for application to human or bio-mimetic motor control. In this poster, we look at promising approaches that can potentially scale and suggest a novel formulation of the actor-critic algorithm which takes steps towards alleviating the current shortcomings. We argue that methods based on greedy policies are not likely to scale into high-dimensional domains as they are problematic when used with function approximation – a must when dealing with continuous domains. We adopt the path of direct policy gradient based policy improvements since they avoid the problems of unstabilizing dynamics encountered in traditional value iteration based updates. While regular policy gradient methods have demonstrated promising results in the domain of humanoid notor control, we demonstrate that these methods can be significantly improved using the natural policy gradient instead of the regular policy gradient. Based on this, it is proved that Kakade’s ‘average natural policy gradient’ is indeed the true natural gradient. A general algorithm for estimating the natural gradient, the Natural Actor-Critic algorithm, is introduced. This algorithm converges with probability one to the nearest local minimum in Riemannian space of the cost function. The algorithm outperforms nonnatural policy gradients by far in a cart-pole balancing evaluation, and offers a promising route for the development of reinforcement learning for truly high-dimensionally continuous state-action systems. Keywords: Reinforcement learning, neurodynamic programming, actorcritic methods, policy gradient methods, natural policy gradien

Empirische Deliberationsforschung

Die deliberative Demokratietheorie ist die wichtigste zeitgenössische Demokratietheorie. Das Versprechen der deliberativen Demokratietheorie, die Legitimation und die Qualität politischer Entscheidungen gleichzeitig zu erhöhen, hat dazu geführt, dass auf allen Ebenen des demokratischen Prozesses diskursive Entscheidungsverfahren unterschiedlicher Art implementiert worden sind. Die wichtige Frage lautet jedoch: Können die empirischen Verfahren die theoretischen Versprechungen wirklich einlösen? Die empirische Diskursforschung bietet hier erste Antworten. In dem Papier diskutieren wir einerseits die Herausforderungen, die aus den empirischen Studien für die Theorie resultieren. Andererseits werden die empirischen Studien selbst auf ihre Aussagekraft hin kritisch befragt. Wir argumentieren, dass die geringe Beachtung, die den Gegenständen der Deliberation bislang geschenkt wurde, ein wichtiger Grund für die disparaten Ergebnisse sein kann, die die empirischen Studien zur deliberativen Demokratietheorie bislang hervorgebracht haben. Es wird darauf hingewiesen, dass der Erfolg einer Deliberation wesentlich davon abhängt, ob die Entscheidungsgegenstände als regulative Fragen, Interessenskonflikte oder ethisch-moralische Fragen geframed werden.The theory of deliberative democracy has strongly influenced philosophical work on democracy over the past twenty years. On at least a theoretical basis, deliberative democracy offers a promising way to improve both the legitimacy and the quality of political decisions at once. This prospect has motivated political scientists and politicians all over the world to implement and analyze a multitude of deliberative forums at all levels of the democratic decision-making process. But the question remains: Can real deliberations fulfill the promise of the theory? Empirical research on deliberation may provide the answer. In our paper, we first discuss the challenges that such studies present to deliberative theory. We then take a critical look at the empirical work that has been done so far on verifying deliberative democratic theory. Comparing different studies on this topic can be quite a frustrating experience, because their results tend to be disparate and sometimes even contradictory. We argue that in order to improve this situation, empirical scientists should pay more attention to the subjects of deliberation. For example, we believe that one of the keys to a successful deliberative process lies in how the topics of the discourse are framed

Family Scents: Developmental Changes in the Perception of Kin Body Odor?

There is increasing evidence that human body odors are involved in adaptive behaviors, such as parental attachment in infants or partner choice in adults. The aim of the present study was to investigate changes in body-odor perception around puberty, a period largely ignored for odor-mediated behavioral changes, despite major changes in social needs and in odor emission and perception. Nine families with two children (8 pre-pubertal, aged 7-10, and 10 pubertal, aged 11-18) evaluated body odors of family members and unfamiliar individuals for pleasantness, intensity, and masculinity, and performed a recognition task. The hypothesized emergence of a parent-child mutual aversion for the odor of opposite-sex family members at puberty was not found, contradicting one of the few studies on the topic (Weisfeld et al., J. Exp. Child Psychol. 85:279-295, 2003). However, some developmental changes were observed, including reduced aversion for odor of the same-sex parent, and increased ability of adults, compared to children, to recognize odor of family members. Sex and personality (depressive and aggressive traits) also significantly influenced odor judgments. Further research with larger samples is needed to investigate the poorly explored issue of how olfactory perception of self and family members develops, and how it could correlate with normal reorganizations in social interactions at adolescenc

Human olfactory communication: current challenges and future prospects

Although anthropologists frequently report the centrality of odours in the daily lives and cultural beliefs of many small-scale communities, Western scholars have historically considered the sense of smell as minimally involved in human communication. Here, we suggest that the origin and persistence of this latter view might be a consequence of the fact that most research is conducted on participants from Western societies who, collectively, were rather old (adults), deodorized and desensitized (ODD) to various aspects of olfactory perception. The view is rapidly changing, however, and this themed issue provides a timely overview of the current state-of-the-art on human chemocommunication. Based on evolutionary models of communication, the papers cover both general mechanisms of odour production by ‘senders’ and odour perception by ‘receivers’. Focus on specific functional contexts includes reciprocal impact of odours between infants and mothers, the role of odour in mate choice and how odours communicate emotion and disease. Finally, a position paper outlines pitfalls and opportunities for the future, against the context of the replication crisis in psychology. We believe a more nuanced view of human chemical communication is within our grasp if we can continue to develop inter-disciplinary insights and expand research activities beyond ODD people

Recht und Internet (Sammelrezension)

Karsten Bremer: Strafbare Internet-Inhalte in internationaler Hinsicht. Ist der Nationalstaat wirklich überholt?Gerhard Hohloch: Recht und Interne